Skip to content

Conversation

@sahar-fehri
Copy link
Contributor

@sahar-fehri sahar-fehri commented Dec 9, 2025

Description

Optimizes TokenListController storage to reduce write amplification by persisting tokensChainsCache via StorageService using per-chain files instead of a single monolithic state property.

Mobile: MetaMask/metamask-mobile#24019

Extension: MetaMask/metamask-extension#39250

Related: https://github.com/MetaMask/metamask-mobile/pull/22943/files

Related: https://github.com/MetaMask/decisions/pull/110

Related: #7192

Explanation

The tokensChainsCache (~5MB total, containing token lists for all chains) was persisted as part of the controller state. Every time a single chain's token list was updated (~100-500KB), the entire ~5MB cache was rewritten to disk, causing:

  • Startup performance issues (loading large state on app initialization)
  • Runtime performance degradation (frequent large writes during token fetches)
  • Impacts both extension

Solution

Per-Chain File Storage:
Each chain's cache is now stored in a separate file (e.g., tokensChainsCache:0x1, tokensChainsCache:0x89)
Only the updated chain (~100-500KB) is written on each token fetch, reducing write operations by ~90-95%
All chains are loaded in parallel at startup to maintain compatibility with TokenDetectionController
Key Changes:

  • Set tokensChainsCache metadata to persist: false to prevent framework-managed persistence
  • Added #loadCacheFromStorage() to load all per-chain files in parallel on initialization
  • Added #saveChainCacheToStorage(chainId) to persist only the specific chain that changed
  • Added #migrateStateToStorage() to automatically migrate existing cache data on first launch after upgrade
  • Updated clearingTokenListData() to remove all per-chain files

References

Checklist

  • I've updated the test suite for new or updated code as appropriate
  • I've updated documentation (JSDoc, Markdown, etc.) for new or updated code as appropriate
  • I've communicated my changes to consumers by updating changelogs for packages I've changed
  • I've introduced breaking changes in this PR and have prepared draft pull requests for clients and consumer packages to resolve them

Note

Moves TokenListController token cache to per-chain persistence via StorageService, reducing write amplification and decoupling it from controller state.

  • BREAKING: tokensChainsCache is no longer state-persisted (persist: false); clients must call await controller.initialize() after construction
  • Per-chain storage keys (tokensChainsCache:<chainId>), parallel load on init, debounced persistence of changed chains only, and robust error handling
  • clearingTokenListData() is async and removes per-chain files; state updated accordingly
  • Keeps deprecated polling APIs but refactors internals; adds extensive tests for migration, persistence, and failure cases
  • Adds @metamask/storage-service dependency and tsconfig references; updates changelog

Written by Cursor Bugbot for commit 3da918d. This will update automatically on new commits. Configure here.

tokensChainsCache: {
includeInStateLogs: false,
persist: true,
persist: false, // Persisted separately via StorageService
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making this false to block disk writes

salimtb
salimtb previously approved these changes Dec 16, 2025
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

this.#persistDebounceTimer = undefined;
}
this.#changedChainsToPersist.clear();
this.#chainsLoadedFromStorage.clear();
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pending state changes lost when controller is destroyed

Medium Severity

The destroy() method cancels the debounce timer and clears #changedChainsToPersist without flushing pending changes to storage. If the controller is destroyed within 500ms of a token fetch or state update, the updated token cache data will be discarded and never persisted. On next app launch, stale data would be loaded from storage instead of the fresher data that was fetched but not persisted.

Fix in Cursor Fix in Web

@sahar-fehri
Copy link
Contributor Author

@metamaskbot publish-preview

@github-actions
Copy link
Contributor

Preview builds have been published. See these instructions for more information about preview builds.

Expand for full list of packages and versions.
{
  "@metamask-previews/account-tree-controller": "4.0.0-preview-a94732c3",
  "@metamask-previews/accounts-controller": "35.0.2-preview-a94732c3",
  "@metamask-previews/address-book-controller": "7.0.1-preview-a94732c3",
  "@metamask-previews/ai-controllers": "0.0.0-preview-a94732c3",
  "@metamask-previews/analytics-controller": "1.0.0-preview-a94732c3",
  "@metamask-previews/announcement-controller": "8.0.0-preview-a94732c3",
  "@metamask-previews/app-metadata-controller": "2.0.0-preview-a94732c3",
  "@metamask-previews/approval-controller": "8.0.0-preview-a94732c3",
  "@metamask-previews/assets-controller": "0.0.0-preview-a94732c3",
  "@metamask-previews/assets-controllers": "96.0.0-preview-a94732c3",
  "@metamask-previews/base-controller": "9.0.0-preview-a94732c3",
  "@metamask-previews/bridge-controller": "64.8.1-preview-a94732c3",
  "@metamask-previews/bridge-status-controller": "64.4.4-preview-a94732c3",
  "@metamask-previews/build-utils": "3.0.4-preview-a94732c3",
  "@metamask-previews/chain-agnostic-permission": "1.4.0-preview-a94732c3",
  "@metamask-previews/claims-controller": "0.4.1-preview-a94732c3",
  "@metamask-previews/composable-controller": "12.0.0-preview-a94732c3",
  "@metamask-previews/connectivity-controller": "0.1.0-preview-a94732c3",
  "@metamask-previews/controller-utils": "11.18.0-preview-a94732c3",
  "@metamask-previews/core-backend": "5.0.0-preview-a94732c3",
  "@metamask-previews/delegation-controller": "2.0.0-preview-a94732c3",
  "@metamask-previews/earn-controller": "11.1.0-preview-a94732c3",
  "@metamask-previews/eip-5792-middleware": "2.1.0-preview-a94732c3",
  "@metamask-previews/eip-7702-internal-rpc-middleware": "0.1.0-preview-a94732c3",
  "@metamask-previews/eip1193-permission-middleware": "1.0.3-preview-a94732c3",
  "@metamask-previews/ens-controller": "19.0.2-preview-a94732c3",
  "@metamask-previews/error-reporting-service": "3.0.1-preview-a94732c3",
  "@metamask-previews/eth-block-tracker": "15.0.1-preview-a94732c3",
  "@metamask-previews/eth-json-rpc-middleware": "23.0.0-preview-a94732c3",
  "@metamask-previews/eth-json-rpc-provider": "6.0.0-preview-a94732c3",
  "@metamask-previews/foundryup": "1.0.1-preview-a94732c3",
  "@metamask-previews/gas-fee-controller": "26.0.2-preview-a94732c3",
  "@metamask-previews/gator-permissions-controller": "1.1.0-preview-a94732c3",
  "@metamask-previews/json-rpc-engine": "10.2.1-preview-a94732c3",
  "@metamask-previews/json-rpc-middleware-stream": "8.0.8-preview-a94732c3",
  "@metamask-previews/keyring-controller": "25.1.0-preview-a94732c3",
  "@metamask-previews/logging-controller": "7.0.1-preview-a94732c3",
  "@metamask-previews/message-manager": "14.1.0-preview-a94732c3",
  "@metamask-previews/messenger": "0.3.0-preview-a94732c3",
  "@metamask-previews/multichain-account-service": "5.1.0-preview-a94732c3",
  "@metamask-previews/multichain-api-middleware": "1.2.6-preview-a94732c3",
  "@metamask-previews/multichain-network-controller": "3.0.2-preview-a94732c3",
  "@metamask-previews/multichain-transactions-controller": "7.0.0-preview-a94732c3",
  "@metamask-previews/name-controller": "9.0.0-preview-a94732c3",
  "@metamask-previews/network-controller": "29.0.0-preview-a94732c3",
  "@metamask-previews/network-enablement-controller": "4.1.0-preview-a94732c3",
  "@metamask-previews/notification-services-controller": "21.0.0-preview-a94732c3",
  "@metamask-previews/permission-controller": "12.2.0-preview-a94732c3",
  "@metamask-previews/permission-log-controller": "5.0.0-preview-a94732c3",
  "@metamask-previews/perps-controller": "0.0.0-preview-a94732c3",
  "@metamask-previews/phishing-controller": "16.1.0-preview-a94732c3",
  "@metamask-previews/polling-controller": "16.0.2-preview-a94732c3",
  "@metamask-previews/preferences-controller": "22.0.0-preview-a94732c3",
  "@metamask-previews/profile-metrics-controller": "3.0.0-preview-a94732c3",
  "@metamask-previews/profile-sync-controller": "27.0.0-preview-a94732c3",
  "@metamask-previews/ramps-controller": "4.1.0-preview-a94732c3",
  "@metamask-previews/rate-limit-controller": "7.0.0-preview-a94732c3",
  "@metamask-previews/remote-feature-flag-controller": "4.0.0-preview-a94732c3",
  "@metamask-previews/sample-controllers": "4.0.2-preview-a94732c3",
  "@metamask-previews/seedless-onboarding-controller": "7.1.0-preview-a94732c3",
  "@metamask-previews/selected-network-controller": "26.0.2-preview-a94732c3",
  "@metamask-previews/shield-controller": "5.0.0-preview-a94732c3",
  "@metamask-previews/signature-controller": "39.0.1-preview-a94732c3",
  "@metamask-previews/storage-service": "0.0.1-preview-a94732c3",
  "@metamask-previews/subscription-controller": "5.4.0-preview-a94732c3",
  "@metamask-previews/token-search-discovery-controller": "4.0.0-preview-a94732c3",
  "@metamask-previews/transaction-controller": "62.9.2-preview-a94732c3",
  "@metamask-previews/transaction-pay-controller": "11.1.0-preview-a94732c3",
  "@metamask-previews/user-operation-controller": "41.0.2-preview-a94732c3"
}

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

@sahar-fehri sahar-fehri enabled auto-merge January 26, 2026 21:28
@sahar-fehri sahar-fehri added this pull request to the merge queue Jan 26, 2026
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jan 26, 2026
@sahar-fehri sahar-fehri added this pull request to the merge queue Jan 26, 2026
Merged via the queue into main with commit e456ae6 Jan 26, 2026
592 of 597 checks passed
@sahar-fehri sahar-fehri deleted the chore/integrate-storage-service-with-tokenListController branch January 26, 2026 22:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants